VIDEO CAMERA, METHOD FOR RECORDING VIDEO AND MACHINE-READABLE MEDIA FOR VIDEO EDITING AND REFORMAT F

专利摘要:
editing and reformatting video for digital video recorder. The present invention relates to some embodiments that provide a video recording device for capturing a video clip. the video recording device receives a selection of a non-temporal compressed encoding scheme from several different encoding schemes to encode the video clip. the different encoding schemes include at least one temporally compressed encoding scheme and at least the selected non-temporally compressed encoding scheme. the video recording device captures the video clip as multiple frames. the video recording device does not temporally encode each of the frames as multiple slices. the slices of a particular frame are for decoding by various processing units of a video decoding device. the video recording device stores the video clip on a storage device.
公开号:BR112012005549B1
申请号:R112012005549-6
申请日:2010-09-09
公开日:2021-08-17
发明作者:Paul Towner；Courtney Kennedy；Randy Ubillos；Greg Mullins；Greg Wallace；Peter Chou；Xin Tong
申请人:Apple Inc；
IPC主号:

专利说明:

FIELD OF THE INVENTION
[001] The invention relates to video recording. Specifically, the invention relates to a video format for a digital video recorder. BACKGROUND OF THE INVENTION
[002] Digital video recorders are commonly used to record digital video for transfer to a computer. Once on the computer, users can edit, enhance and share digital video. However, today's digital video recorders compress digital video using forms of encoding that use temporal compression. That is, compressed video includes predictive (P) and bidirectional (B) frames that are not real images, and instead are just mathematical data that represents the difference between an index frame of (I) that is encoded as an image. .
[003] Temporal compression allows compression of digital video to smaller camera file sizes, but creates a multitude of problems for users who want to transfer the video to their computers in order to work with the video. Because P and B frames are defined only by reference to other frames, they must be decoded for a user to edit them. This transcoding usually takes place after importing the digital video from the camera.
[004] Figure 1 illustrates a prior art system with a video camera 105 and a computer 110. The video camera 105 captures and stores a video file 115 having a size X. This video is encoded using temporal compression. After transferring from camera 105 to computer 110, the video must be transcoded (to remove temporal compression) and stored. The resulting file 120 is 3X to 10X in size, and therefore is much larger than the original file in the camera. Due to these expansions, video for file size need not become prohibitive for most users.
[005] Furthermore, transcoding is a computation and time intensive process. Transferring 30 minutes of video may take 90 minutes due to transcoding. Therefore, there is a need for a video camera with video recording capability that is not temporally compressed without sacrificing quality or creating excessively large file sizes. SUMMARY OF THE INVENTION
[006] Some embodiments of the invention provide a video recording device (eg, a video camera) that captures and stores digital video in a format that is not temporally compressed. The captured digital video is stored at a particular desired resolution and/or bit rate while maintaining a desired video quality.
[007] When digital video is exported from the recording device to a computer (eg for sharing, editing, etc.), the video is transferred quickly, with no need for transcoding. Transcoding, in some embodiments, involves decoding video after import to remove any temporal compression and then re-encoding the video without temporal compression. As such, when video does not need to be decoded, digital video is stored on the computer in its native format.
[008] In some embodiments, the video recording device provides users with an option of storing video that is either temporally compressed or non-temporally compressed. Temporal compressed video includes encoded interframe video images (e.g., frames) that are encoded at least partially by reference to one or more video images. Temporally uncompressed video includes only intraframe encoded video images (e.g., frames) which are encoded without reference to any video images.
[009] Some embodiments include non-temporal compressed high definition and/or enhanced definition formats at a manageable bit rate. The various video formats are presented through a digital video recorder user interface. In some modalities, the many different video formats all use the same encoding standard. That is, the temporally compressed and the non-temporal compressed formats use the same encoding standard.
[0010] Some modalities provide a media editing application with the ability to recognize the incoming video format. When incoming digital video (for example, from a video recording device as described above) is temporally compressed, the media editing application transcodes the digital video. When digital video is not temporally compressed, the media editing application stores the video without transcoding or expanding the video size. Thus, non-temporal compressed digital video can be imported very quickly because there is no transcoding. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The novel features of the invention are defined in the appended claims. However, for purposes of explanation, various embodiments of the invention are defined in the following figures.
[0012] Figure 1 illustrates a prior art system with a video camera and a computer.
[0013] Figure 2 illustrates a system of some modalities that includes a digital video camera and a computer.
[0014] Figure 3 illustrates a sequence of digital video images that are encoded using temporal compression.
[0015] Figure 4 illustrates a sequence of digital video images that is encoded without using temporal compression.
[0016] Figure 5 illustrates a user interface of a video camera of some modalities that allows a user to select a video format option for a captured video.
[0017] Figure 6 illustrates a user interface of a video camera of some modalities that allows the user to specify bit rate settings of a captured video.
[0018] Figure 7 illustrates the software architecture of a digital video camera of some modalities for capturing, encoding and storing digital video.
[0019] Figure 8 conceptually illustrates a process of some modalities for capturing and storing video in a digital video camera that has the ability to store video compressed in a temporal way or compressed in a non-temporal way.
[0020] Figure 9A conceptually illustrates an example of video resolution width reduction of some of the modalities.
[0021] Figure 9B conceptually illustrates an example of increased video resolution width of some modalities.
[0022] Figure 10 conceptually illustrates the performance of transformed into 8x8 macroblocks during video encoding of some modalities.
[0023] Figure 11 conceptually illustrates the direction of prediction for the intraprediction modes of some modalities.
[0024] Figure 12A conceptually illustrates video images encoded on a slice-by-slice basis of some modalities.
[0025] Figure 12B conceptually illustrates video images decoded on a slice-by-slice basis of some modalities.
[0026] Figure 13 illustrates a process of some modalities for performing non-temporal encoding of a video.
[0027] Figure 14 conceptually illustrates a process for defining different video formats for a video recording device of some modalities.
[0028] Figure 15 illustrates a block diagram of a video camera of some modalities that uses the video capture, encoding and storage process of Figure 8.
[0029] Figure 16 illustrates a media editing application of some modalities for importing and editing digital video that has the ability to differentiate between different incoming digital video formats.
[0030] Figure 17 conceptually illustrates a process of some modalities for storing a video clip imported to a computer from a digital video source.
[0031] Figure 18 illustrates a computer system with which some embodiments of the invention are implemented. DETAILED DESCRIPTION OF THE INVENTION
[0032] In the following description, numerous details are presented for purposes of explanation. However, one of skill in the art will understand that the present invention can be practiced without the use of these specific details. For example, some of the examples illustrate specific coding modules. One skilled in the art will recognize that different encoding modules are possible without departing from the invention.
[0033] Some embodiments of the invention provide a video recording device that captures and stores digital video in a format that is not temporally compressed. The captured digital video is stored at a particular desired resolution and/or bit rate while maintaining a desired video quality. When digital video is exported from the camera to a computer, the digital video is stored on the computer in its native format, without transcoding.
[0034] Figure 2 illustrates a system, including a digital video camera 205 and a computer 210. The digital video camera captures and stores a video file 215 that has a Y size. The video file 215 is compressed in a manner not temporal. That is, each digital video image (ie, frame or field) in the video file is encoded without reference to other digital video images. Figures 3 and 4, described below, illustrate different types of frames. The non-temporal compressed video clip is transferred (for example, via USB, FireWire, or other wired or wireless connection) from the camcorder 205 to the computer 210. As described below, the computer 210 may include a media editing app for video editing and augmentation. Computer 210 stores the video clip in its native video file format 220. This video file 220 is the same Y size as the camera's video file 215.
[0035] No transcoding needs to be performed on the import, as there is no temporal compression to remove. Not only does this result in the file having the same size, but the transfer time is only limited by the file size and the speed of the connection between camera 205 and computer 210. When transcoding must be performed, faster transfer promises which is supposed to come with the camera's random access storage (ie hard drives, flash memory, etc.) is nullified by the slow transcoding process.
[0036] As mentioned above, the video recording device of some modalities stores digital video in a format that is uncompressed and temporal form. Non-temporal compressed video includes only intraframe encoded digital video images (e.g., frames) which are encoded without reference to any other digital video images. By comparison, Figure 3 illustrates a sequence of 300 digital video images that is temporally compressed. Temporally compressed video includes interframe encoded digital video images (e.g., frames) that are encoded at least partially by reference to one or more video images. Figure 3 illustrates I frames (intraframes that are not coded with reference to any other frames), P frames (predictive frames that are coded with reference to previous frames), and B frames (bidirectional frames that are coded with reference to previous and futures).
[0037] Sequence 300 includes an I-frame, then two B-frames, then a P-frame, then two more -B-frames, etc. The sequence from I-frame 305 through the fifteenth total frame is known in some embodiments as a group of pictures (GOP). In this case, the size of the GOP is 15. Each GOP starts with an I-frame.
[0038] Some modalities, instead of using -I, -P, and -B frames for temporal compression, use -I, -P, and -B slices for temporal compression. Each digital video image (e.g., frame) of some embodiments includes numerous macroblocks, each of which is a 16x16 array of pixel values. A slice is a group of consecutive macroblocks. Rather than determining how to encode macroblocks on a picture-to-picture basis, some modalities make this decision on a slice-by-slice basis instead. Also, when decoding a video image that has been encoded on a slice-by-slice basis, each slice can be decoded independently of one another.
[0039] Figure 4 illustrates the case where a video frame 400 is not temporally compressed. Instead, each video frame in sequence 400 is an -I frame, defined without reference to the other frames. Although this format is not as compressed in-camera as sequence 300, sequence 400 does not need to be decoded when transferring to a computer and can be edited much more easily than a compressed temporal sequence.
[0040] Some modalities provide a media editing application with the ability to recognize the incoming digital video format. The media editing application transcodes digital video only if the video is temporally compressed. When digital video is not temporally compressed, the media editing application stores the video without transcoding or expanding the video size. I. Digital Video Camera
[0041] As noted above, some embodiments provide a video recording device (eg, a digital video camera) that captures and stores the digital video and a format that is non-temporal compressed. Some modalities provide users with the option of recording video that is temporally compressed or non-temporally compressed. This option is presented in a video camera user interface in some modalities. A. User Interface
[0042] Figure 5 illustrates a video camera user interface that allows the user to select a video format option for a captured video. Specifically, this figure shows the video camera user interface in two different phases: the first phase which is before a user's selection of the iFoto video format option and a second phase which is after their selection. As shown, video camera 500 includes a 505 user interface with a 510 display screen for displaying a graphical user interface (GUI) that includes a 515 menu. The graphical user interface can be entirely textual, entirely graphical, or a combination of these. The 505 user interface also includes several 520 and 525 user selectable controls.
[0043] Menu 515 displays a list of video format options. These options include several options for iframes (ie, not temporally compressed) at different resolutions (ie, iFrame 960x540, iFrame 1280x720) and several options for temporally compressed format. The format options in the menu range from high definition to enhanced definition, however the menu may exclude one or more options or include other options (eg iQuadro 640x480). Some modalities only include a non-temporal compressed option (eg 1280x720).
[0044] As mentioned above, some modalities provide a 960x540 iquadro recording format option. This recording format has a vertical resolution of 540p. This resolution is advantageous for a number of reasons, one of which is that the resolution often matches the camera's native sensor resolution and can be easily upscaled (eg by a computer used to edit the video) by standards of HD, such as 720p, 1080i or 1080p.
[0045] User selectable controls 520 and 525 on the camcorder allow a user to navigate menu 515. In particular, control 520 is for navigating menu 515 vertically, while controls 525 are for navigating the menu horizontal. In the example illustrated in Figure 5, these controls are provided as physical controls on the camcorder. However, in some embodiments, such navigation controls may be provided as part of the graphical user interface presented on a display screen. Alternatively, or together, the 500 camcorder can be equipped with a touchscreen that allows the user to directly select a video format option using the touchscreen, without having to use such physical controls as the 520 controls. and 525.
[0046] The operation of the user interface will now be described by reference to two different phases that are illustrated in figure 5. In the first phase, the display screen 510 shows menu 515. The selected recording format is a temporally compressed option (1080p). A camcorder user interacts with the 515 menu through the 520 and 525 controls. Specifically, the user selects the top control of the 520 controls in order to move up the option selected by a menu item and change the format option of video from a temporally compressed format to an iPicture format.
[0047] As shown in two phases, once the user selects the top control from the 520 controls, the 515 menu highlights the iFoto format option (ie iFoto 960x540). This highlight provides the user with a visual indication of the selection of the iTable format option. Now that the user has selected the iPicture format option, video clips captured later will be recorded without temporal compression at the specified resolution.
[0048] In the previous example, menu 515 displays a list of options of different video formats for encoding a sequence of video frames captured using different encoding schemes and resolution. In some embodiments, menu 515 displays one or more other options for specifying other encoding formats. Figure 6 illustrates the 505 user interface, which allows the user to specify not only the resolution and encoding scheme, but also the bit rate. The video bitrate, in some modalities, is the video file size per playing time. In general, the higher bitrate will lead to higher video quality if the resolution is kept the same. However, higher bitrates also mean larger files, which can be tricky for a user to work with.
[0049] This value is similar to the previous figure, however, the 515 menu exposes multiple iPicture format options at the same resolution with different bit rate settings. Specifically, menu 515 displays two different bitrate settings (ie, two 24Mbps, 20Mbps, or 16Mbps) for each of the two iquadro resolutions (ie, iquadro 960x540, iquadro 1280x720). As shown, without changing the iPicture resolution, the user selects the 520 controls background control to change the bit rate setting from 24 Mbps to 16 Mbps. In some embodiments, a media editing application that the camera will eventually transfer video to has a specified maximum bit rate (eg 24 Mbps). In some embodiments, the video camera menu 515 may allow a user to select video encoding options. For example, the 515 menu may present selectable frame rate options (for example, 25 or 30 frames per second). B. Architecture
[0050] Figure 7 illustrates the software architecture of a 700 digital video camera for capturing, encoding and storing digital video. Digital video camera 700 includes a user interface 705, a video compression controller 710, a discrete cosine transform (DCT) unit 715, a quantizer unit 720, an entropy encoder 725, an inverse unit of the quantizer 730, an inverse discrete cosine transform (IDCT) unit 735, a motion compensation, motion estimation, and intraframe prediction unit 740, an adder 745, and an image capture unit 750.
[0051] The camera also includes a storage for compression settings 755 and a video storage 760. In some embodiments, the two storages are the same physical storage. In other embodiments, the two storages are separate physical storage on the camera or are different partitions of the same physical storage. Video storage 760 is digital tape in some modes. In other embodiments, video storage 760 is non-volatile storage, such as magnetic disk storage (eg, hard disk) or solid-state memory (eg, flash memory). In some embodiments, video storage 760 is random access storage such as flash memory. Examples of flash memory include Secure Digital (SD), CompactFlash (CF), Memory Stick (MS), and other types of flash memory. When storage 760 is random access storage, a user (for example, a user of a computer to which the video camera is attached) can choose to access a second video clip before a first video clip, even if the second video clip is recorded after the first video clip. Different modalities of video storage 760 can be configured to store data (eg captured digital video) using different file systems such as File Allocation Table (FAT), Hierarchical File System (HFS), File Allocation Table extended (exFAT) and other different types of file systems.
[0052] The 700 camera user interface 705 includes both the graphical user interface, as illustrated in display 510 in the previous figures, as well as user input controls such as controls 520 and 525 illustrated in the same figures. The graphical user interface can be a text-only interface or it can include a graphic as well.
[0053] As illustrated above, the user input format selects information through the 705 user interface. By choosing a compression type (temporal or non-temporal), a resolution, and/or a bit rate, the user determines the format to subsequently the recorded video. This 765 format selection information is transferred from the user interface to the video compression controller 710.
[0054] The video compression controller 710 instructs the various compression and encoding modules how to perform encoding to the specified format. The video compression controller extracts compression settings from 755 storage based on the selected format. These compression settings are then transferred to the various compression and encoding modules so that they can properly encode the video in the specified format. Figure 7 illustrates that the video compression controller instructs the DCT unit 715, the quantizer unit 720, the entropy encoder 725, and the motion estimation, motion compensation, and intraframe prediction unit 740. , information similar to that given for the DCT and quantizer units 715 and 720 is also passed to invert the IDCT and quantizer units 730 and 735.
[0055] Image capture unit 750 captures video. For more details on the video capture process, see Figure 15 below. In some modalities, video is captured at a rate of 25 or 30 frames per second. This is a user option in some modes and a non-variable setting in other modes. Each captured frame is essentially an image captured by the video camera. A captured frame is sent from the imager to compression and encoding modules 715 to 745 so that the frame can be encoded.
[0056] The DCT unit 715 performs discrete decosine transforms in image data blocks resulting from the addition or subtraction performed in the adder 745. The discrete cosine transform operation achieves compression, removing some spatial redundancy that exists within a block of data from Image. The operation transforms an image data block into a two-dimensional array of DCT coefficients in which most of the block's energy is typically concentrated into a few low-frequency coefficients.
[0057] The quantizer unit 720 applies quantization on the DCT coefficients produced by the DCT unit 715. The quantization operation achieves compression of the DCT coefficients by compressing a range of values to a single quantum value. Quantization causes quality loss and thus some modalities use a quantization matrix to minimize image quality loss by assigning small quantization steps to certain frequencies of DCT coefficients.
[0058] The 725 entropy encoder converts input data to variable length codes. In some embodiments, input data comes directly from the quantizer unit 720. In other embodiments, intermediate operations such as zig-zag scanning and run-length encoding are performed between the quantizer unit 720 and input encoder - sink 725. The 725 entropy encoder, in some embodiments, achieves compression by assigning the code of shorter length words to values that have a higher probability of occurring than to values that have a lower probability of occurring (for example, Context-based Adaptive Variable Length Coding). Some embodiments use encoding systems such as Huffman or UVLC where entropy encoding is performed over a symbol-based symbol. Other modalities use encoding schemes such as arithmetic encoding where an entire block of data is encoded as a single number (eg, context-based Adaptive Binary Arithmetic Encoding). The entropy encoder generates an encoded frame, which can be stored in video storage 760.
[0059] Some modalities perform spatial or temporal prediction to achieve additional compression of video images. To facilitate this, some arrangements include a video decoding path so that the encoder can use the same decoded reference frames used by a decoder to perform the prediction. The decoding path includes the inverse quantizer unit 730 and the inverse DCT unit 735; these units perform the reverse operations of the quantizer unit 720 and the DCT unit 715 as described above.
The motion estimation, motion compensation, and intraframe prediction unit 740 perform motion estimation, motion compensation, and intraframe prediction operations. The motion compensation operation is part of the decoding path, which uses the temporal prediction information to compensate the output of the inverse DCT unit 735 in order to reconstruct and decode a video image. The motion estimation operation is part of the coding path; it searches other decoded frames for a pixel matching block to create motion vectors for use in temporal prediction. Intraframe prediction has an encoding component and a decoding component. The decoding component of the intraframe prediction operation uses the spatial prediction information to reconstruct and decode a video image. The encoding component of the intraframe prediction operation searches for the current frame decoded by a pixel coupling block for use in spatial prediction. In some embodiments, unit 740 will only perform spatial intraframe prediction when instructed not to perform temporal compression.
The adder 745 computes the difference between the image from the image capture unit 750 and the output of motion estimation, motion compensation and intraframe prediction unit 740. The resulting difference (or summation) is then sent to DCT 715 unit to be encoded as mentioned above.
[0062] The operations of each of the DCT, quantizer, and entropy encoder units 715 to 725 are determined by numerous different variables. Each of these variables can be set differently depending on the specified video format. Thus, the DCT operation is controlled not by a certain configuration in some modalities, but by a multitude of different choices. In some embodiments, these are design choices by the camera manufacturer that are intended to maximize video quality at a particular resolution and bit rate. Likewise, the quantizer and entropy encoding operations are also controlled by a multitude of different choices in some modalities that are design options for each special format designed to maximize video quality at the special resolution and bit rate. For example, the quantization matrix used by the quantizer can be modified based on the video format.
[0063] When the video compression controller 710 specifies settings for non-temporal compressed video, the motion estimation, motion compensation, and intraframe prediction unit 740 are instructed to only perform intraframe prediction, instead of the estimation of motion and motion compensation operations that are part of temporal compression. On the other hand, when the video compression controller specifies settings for temporally compressed video, the unit 740 performs motion estimation and motion compensation in addition to intraframe prediction.
[0064] Furthermore, in some embodiments the video decompression controller 710 performs rate control during the encoding process in addition to specifying encoding variables for the different modules. To perform rate control, controller 710 calculates, after encoding each frame, the proximity of the encoded video image to a target bit rate (that is, the bit rate specified for the video format). Controller 710 then sets the compression variables (for example, DCT unit 715 and quantizer unit 720 variables) as direct copy (on the fly) to adjust the size of the frame to be encoded. In some embodiments, how these changes are made is part of the compression settings indicated by the selected video format.
[0065] Although many of the features of camera 700 have been described as being realized by one module (eg, the video compression controller 710), one skilled in the art will recognize that functions can be divided into multiple modules, and performance of a resource may even require multiple modules. Likewise, features that are shown as being performed by separate modules can be performed by one module in some modalities.
[0066] Figure 8 conceptually illustrates a process 800 of some modalities for capturing and storing video in a digital video camera that has the ability to store any temporally compressed or non-temporally compressed video (for example, the camera 700). The 800 process begins by identifying (at 805) a video format selected for the captured video that specifies a particular resolution and/or bit rate. This video format is selected by a user in some modes through a user interface, as illustrated in figures 5 and 6.
[0067] Process 800 determines (in 810) whether to perform temporal compression on a video clip that is currently to be captured. This determination is made based on the selected video format. When the user has selected an iPicture recording mode, no temporal compression is performed. On the other hand, when the user has selected a different recording mode (eg AVC HD 1080p), temporal compression is required.
[0068] When temporal compression is required, the process receives (in 815) the next captured video image. The process then compresses (at 820) the video image both spatially and temporally and encodes (at 820) the video image. This operation is performed by coding several modules from 715 to 745 in some modalities. The process then stores (in 25) the encoded video image in a camera storage. Process 800 then determines (at 830) if the camera is still capturing the video (that is, if there are more frames of any video to compress and encode). When the camera is no longer capturing video, the process ends. Otherwise, the process returns to 815 to receive the next captured video image.
[0069] When temporal compression is not required for the currently captured video clip, the process receives (at 835) the next captured video image. The process then compresses (at 840) the video image spatially. In some embodiments, this operation is performed by unit 740, although only intraprediction is used. Since the video image is not being temporally compressed, no motion estimation or motion compensation needs to be performed.
[0070] Next, process 800 performs (in 845) a discrete cosine transform on the video image using variable according to the selected format. That is, the discrete cosine transform is performed using variables sent to the discrete cosine transform unit 715 by the compression video controller 710, in some embodiments. These variables are selected (in some modalities, such as a design choice by the camera manufacturer) to produce high quality video products with a desired resolution and/or bit rate without performing temporal compression on the video.
[0071] The process then quantifies (at 850) the video image (output from the DCT unit) using variables according to the selected format. That is, quantization is performed using variables sent to quantizer unit 720 by video compression controller 710, in some embodiments. These variables are selected (in some modalities, such as a design choice by the camera manufacturer) to produce high quality video products with a desired resolution and/or bit rate without performing temporal compression on the video.
[0072] The process then entropy encodes (in 855) the video image (the output of the quantizer unit and any intermediate modules such as a run-length encoder) using variables according to the selected format. That is, entropy encoding is performed using variables sent to entropy encoder 725 by video compression controller 710, in some embodiments.
[0073] These variables are selected (in some embodiments, such as a design choice by the camera manufacturer) to produce high quality video products with a desired resolution and/or bit rate without performing temporal compression on the video.
[0074] Process 800 below stores (at 860) the encoded video image in a camera store. Process 800 then determines (at 865) if the camera is still capturing the video (that is, if there are more frames of any video to compress and encode). When the camera no longer captures video, the process ends. Otherwise, the process returns to 835 to receive the next captured video image.C. Exemplary Video Processing Setup
[0075] The above section describes various aspects of a video recording device of some modalities that captures and stores digital video in a format that is not temporally compressed. The following section describes aspects of a non-temporal compressed video processing setup of some different modalities and processes performed by such setup. In some embodiments, the setting is determined based on design choices by the camera manufacturer that are designed to maximize video quality at a particular resolution and bit rate and to decrease video decoding time, among other design goals. .
[0076] Different video processing techniques can be employed to increase the video quality (or decrease the bit rate) of non-temporal compressed video. One technique is to reduce the width of the video resolution from a selected resolution (eg iPicture 1280x720) before the video is encoded. As such, some modalities of the exemplary configuration reduce the downsampling of the video width in order to increase the video quality or and/or reduce the encoded video bit rate.
[0077] Figure 9A conceptually illustrates an example of video resolution width reduction of some modalities. In this example, it is assumed that the selected recording option is iPicture 1280x720. Figure 9A shows video images (e.g., image 925 and image 910) at various stages of a video processing operation performed by a scaler 905 and an encoder 910. In some embodiments, the encoder 910 may be implemented by an encoder 1520 of Figure 15, described below. As shown, the uncoded video images, which have a resolution of 1280x720 (eg the 925 image), are inserted into the 905 scaler. The 905 scaler reduces (i.e., scales) the resolution width of the video images, which now have a resolution of 960x720 (as shown for image 930), and output the video images for encoder 910 to be encoded. In some embodiments, when scaler 905 reduces the resolution width of video images, it also transversely extends the pixels so that the video aspect ratio is maintained. This is illustrated in Figure 9A by describing the pixels of image 925 as squares and the pixels of image 930 as rectangles.
[0078] Some modalities of exemplary configuration also increase the video resolution width after being decoded. For example, some such modalities increase the video resolution width back to the width at which the video was originally captured. Figure 9B conceptually illustrates an example of the increased video resolution width of such modalities. Figure 9B shows video images (e.g., image 935 and image 940) at various stages of a video processing operation performed by a decoder 915 and a scaler 920. In some embodiments, the decoder 915 may be implemented by decoded video 1540 of Figure 15, described below.
[0079] Continuing with the example illustrated in Fig. 9A, when the video images encoded in Fig. 9A are to be decoded (eg by media editing application 1600, described below), they are sent to decoder 915. video images are decoded, the scaler 920 increases (i.e., scales) the resolution width of the video images before the video images are stored and/or viewed. As shown by image 935, the decoded video images have a resolution of 960x720 before the climber processes them. Since the video images were originally captured at a resolution of 1280x720, the 920 scaler increases the resolution width of the decoded video images back to the 1280x720 resolution, as illustrated by the 940 image. In some embodiments, when the scaler 920 increases the resolution width of the video images as well as reducing the pixels transversely so that the video aspect ratio is maintained. Figure 9B illustrates this by describing the pixels of image 935 as rectangles and the pixels of image 940 as squares.
[0080] The above example describes the video that is captured at the selected full recording resolution (ie the image sensor of the video recording device captures video images at a resolution of 1280x720). In some modalities, the video recording device does not capture the full selected recording resolution. Rather, these modalities capture video at a resolution that is less than the selected video recording resolution and perform resolution width boost after the video is decoded. For example, referring to the example above, some such modalities capture video at a resolution of 960x720. When video is decoded, the video resolution is increased to 1280x720 as illustrated in figure 9B.
[0081] Although the example illustrated in Figures 9A and 9B shows video scaling images captured with a resolution of 1280x720 to 960x720 before the video images are encoded and scaling the video images from 960x720 back to 1280x720 after the images For video images to be decoded, different modalities scale the video images to different sizes. For example, some modalities can capture video at a resolution of 1920x1080, scale the video images to 960x720 before encoding the video images, and scale the video images to 1280x720 after the video images are decoded. Other modalities cannot scale video images after video images are decoded.
[0082] In addition, not only do some modalities scale the resolution width of video images, but also scale the resolution height of video images. Some modalities scale video images, while other modalities proportionally scale video images anaphorically. In some modalities, determine whether to scale video images before encoding and after decoding, to what resolution the video images are scaled before being encoded, and to what resolution video images are scaled after being decoded (eg when video is played on the video recording device itself) are design choices made by camera manufacturers that are aimed at maximizing video quality at a special resolution and bitrate. In addition, some modalities insert metadata into a video image that indicates its original prescale resolution so that a video playback device (eg, a computer, smartphone, etc.) can use this information in scaling the images. of video for playback.
[0083] An additional method of reducing the bit rate (or increasing the video quality) of non-temporal compressed video is to perform transforms on an 8x8 macroblock basis during video encoding. As shown above in Figure 7 and described in the relevant sections, the video captured by the image capture unit 750 is processed on a frame-by-frame basis (ie, frame-by-frame basis). In particular, the DCT 715 processes video images on a frame-by-frame basis. That is, DCT 715 receives image data for an entire frame from adder 715 and performs a discrete cosine transform on the image data, which is then output to quantizer 720.
[0084] In some embodiments of the exemplary configuration, aoDCT 715 is configured to perform discrete cosine transform on 8x8 sized macroblocks. Figure 10 conceptually illustrates performing 8x8 macroblock transforms during video encoding. As shown, DCT 715 receives image data for an 8x8 macroblock in the form of an 8x8 matrix from adder 715, performs a discrete cosine transform on the image data, and generates the transformed data data for the quantizer. 720 (not shown). However, other embodiments of the exemplary configuration can perform discrete cosine transforms in different sizes, such as 16x16 macroblock.
[0085] Yet another method of reducing the bit rate (or increasing the video quality) of non-temporal compressed video is to perform all intraprediction modes when encoding the video. Some embodiments of the exemplary configuration use the conventional H.264 codec standard for intraprediction encoding the video images, while other embodiments of the exemplary configuration do not perform any intraprediction encoding. Instead, these modalities encode the video using only intra-mode encoding. In some embodiments, intra mode encoding is encoding without using any intraprediction modes.
[0086] As discussed above, in some embodiments, a video is made up of a sequence of video frames where each frame is composed of multiple macroblocks. A macroblock is typically a 16x16 pixel matrix (although other macroblock sizes are also possible, such as the 8x8 macroblocks discussed above) and is divided into partitions (such as the 4x4 pixel matrix partitions). Under the H.264 codec standards, when intrapredicting that encodes a frame, there are nine different ways to encode a 4x4 matrix (ie, there are nine 4x4 intrapredict modes). The nine modes are:0. Intra_4x4_Vertical1. Intra_4x4_Horizontal2. Intra_4x4_DC3. Intra_4x4_Diagonal_Down_Left4. Intra_4x4_Diagonal_Baixo_Right5. Intra_4x4_Vertical_Direita 6. Intra_4x4_Horizontal_Baixo7. Intra_4x4_Vertical_Left8. Intra_4x4_Horizontal_Up
[0087] Each 4x4 matrix is encoded in a unique prediction mode. Typically, the prediction mode that results in the least cost will be selected. The cost is typically equal to distortion (where distortion reflects the difference between the original pixel values and encoded predictive values) or the weighted average of distortion and a number of bits produced by the prediction mode (where an increase in distortion and/ or in the number of bits increases the cost). An exhaustive search among all nine prediction modes can be performed to determine the optimal prediction mode (the selected prediction mode) having the lowest cost. Some modalities apply heuristic search to select an intraprediction mode instead of performing an exhaustive search among the nine prediction modes.
[0088] Figure 11 conceptually illustrates the prediction directions for the nine intraprediction modes mentioned above. For a currently processed 4x4 matrix, the prediction modes under the H.264 standard indicate the position (relative to the currently processed 4x4 matrix) of another 4x4 matrix (herein referred to as the predictive matrix), which is to be the basis of the encoded prediction information for the currently processed matrix. For example, predictive mode 0 (vertical) indicates that the predictive matrix for a currently processed matrix is located above the currently processed matrix and predictive mode 1 (horizontal) indicates that the predictive matrix for a currently processed matrix is located to the left of the matrix currently processed.
[0089] In addition, there are several ways to decrease the decoding time of temporally uncompressed video. As noted above, when a video image has been encoded on a slice-by-slice basis, each slice of the video image can be decoded independently of the other slices. Therefore, one way to decrease the decoding time of non-temporal compressed video is to encode the video images on a slice-by-slice basis so that a computing device having multiple processors can independently and simultaneously decode the video images. slices of the video images.
[0090] Figure 12A conceptually illustrates video images encoded on a slice-by-slice basis of some modalities. As shown, this figure shows video images 1200 that have not been processed by an encoder 1205. In some embodiments, encoder 1205 may be implemented by the video encoder 1520 of Figure 15, described below. In this example, encoder 1205 is configured to divide the video images into four slices and encode the video images on a slice-by-slice basis. Image 1210 conceptually illustrates a video image that has been split into four slices. As illustrated, image 1210 is divided into four sections 1215 through 1230. However, the manner in which sections 1215 through 1230 are divided is for illustrative purposes only. Other embodiments of encoder 1205 can divide image 1210 into any number of sections, and into any number of different shapes and sizes.
[0091] Since each slice of a video image encoded on a slice-by-slice basis can be independently decoded, a computing device (eg smartphone, computer, etc.) with multiple processors can decode multiple slices while Same time. Figure 12B conceptually illustrates the hardware decoding of a computing device. This figure shows decoding hardware 1235 that includes a processing system 1240 and a decoding controller 1290. In some embodiments, the decoding hardware 1235 may be implemented by the video decoder 1540 of Fig. 15, described below.
[0092] Processing system 1240 includes four processing units 1245 to 1260 (processors, cores, processing cores, etc.). In some embodiments, the processing units 1245 to 1260 are implemented as cores in a single matrix while in other embodiments the processing units 1245 to 1260 are implemented as chips of a single package. In still other embodiments, processing units 1245 to 1260 are implemented as multiple packages in a single system. The decoding controller 1290 of some embodiments is responsible for allocating or assigning slices of a video image to processing units 1245 to 1260 for decoding. As such, when decoding hardware 1240 identifies that a video frame is encoded on a slice-by-slice basis and identifies the video frame's slices, the decoding allocates controller 1290 or assigns the identified slices to one or more of the units from processing 1245 to 1260 for decoding. In some embodiments, decode controller 1290 may be implemented by one or more of the processing units 1245 to 1260 while in other embodiments, decode controller 1290 may be implemented by a separate hardware component.
[0093] Continuing with the example illustrated in Fig. 12A, the video images, which have been split into four slices and encoded on a slice-by-slice basis, are sent to the decoding hardware 1235 for decoding. The slices of each video image are identified and assigned to a processing unit of processing system 1240 for decoding. As shown, video image 1265 includes four slices 1270 to 1225. Decode controller 1290 assigns slice 1270 to processing unit 1245, slice 1275 to processing unit 1250, slice 1220 to processing unit 1255, and slice 1225 to processing unit 1260. As each slice can be decoded independently of the others, assigning a different processing unit from processing system 1240 to each of slices 1270 to 1225 allows for slices 1270 to 1225 of video image 1265 are independent and simultaneously decoded. Thus, this decoding technique results in decreased decoding time in some modalities.
[0094] In Fig. 12B, each slice in a video image is simultaneously independently and decoded by a single processor of processing system 1240. However, different modalities can allocate or assign different numbers of processors to decode a slice of the video images. For example, some embodiments may assign two processing units of processing system 1240 to decode each slice of the video image. As such, at most two slices of a video image can be simultaneously decoded at any one time by the processing system of illustrated 1240. Other embodiments may assign all processing units 1245 to 1260 of processing system 1240 to decode each slice of the video image. In such modalities, only one slice can be decoded at a time. Determining the number of slices the video images are divided into and the shape and size of the slices are choices made by camera manufacturers with the intention of reducing video decoding time.
[0095] Figure 13 illustrates a process 1300 of some embodiments of the exemplary configuration for non-temporal encoding a video. In some embodiments, motion estimation, motion compensation, and the intraframe prediction unit 740 described above perform at least a part of process 1300. In some embodiments, process 1300 may be performed by video encoder 1520, described below.
[0096] For a video image in video, the process starts by reducing (in 1305) the width of the video image resolution. Operation 1305 can be performed as described above with reference to Figure 9A. Process 1300 then divides (at 1310) the video image into a number of slices as discussed above with reference to Fig. 12A. Process 1300 then selects (at 1315) a macroblock from one of the slices. Then, process 1300 selects (at 1320) an intraprediction mode for the selected macroblock and executes (at 1325) selected prediction mode. Process 1300 then calculates (at 1330) a cost for the selected intraprediction mode as described above.
[0097] Next, the process determines (in 1335) whether any unselected intraprediction modes are left. If the unselected intrapredict modes remain, the process returns to operation 1320 to continue executing the other unselected intrapredict modes. If there is no unselected intraprediction mode left, process 1300 selects (in 1340) the intraprediction mode with the lowest calculated cost.
[0098] In 1345, process 1300 determines if the selected intra-prediction mode is good enough. In some embodiments, the selected intraprediction mode is good enough when the overall cost of encoding the macroblock using the selected intraprediction mode is less than the cost of encoding the macroblock using the intramode. The cost of encoding the macroblock using the selected intraprediction mode includes an overhead cost that is not included in the intramode. In some embodiments, encoding the intra macroblock using the macroblock encoding mode is without using any intraprediction modes.
[0099] If process 1300 determines that the selected intraprediction mode is not good enough, process 1300 selects (at 1350) intra mode as the encoding mode for the macroblock. However, if process 1300 determines that the selected intraprediction is good enough, process 1300 selects (at 1355) the selected intraprediction mode as the encoding mode for the macroblock. After determining the coding mode for the macroblock, the process performs (1300 to 1360) an 8x8 transform on the macroblock, as described above with reference to Fig. 10.
[00100] Finally, the process determines (at 1365) whether any unselected macroblocks in the slice remain. If there are unselected macroblocks on the left slice, the process returns to operation 1315 and processes another macroblock on the slice. Therefore, process 1300 repeats operations 1315 to 1360 until there are no more macroblocks left in the slice, at which point, the process ends. Although process 1300 describes macroblock processing for a slice of the video image, operations 1315 to 1365 are performed for each slice in the video image.
[00101] Figure 14 conceptually illustrates a process for defining different video formats for a video recording device of some modalities. The video recording device provides both temporally compressed and non-temporally compressed video formats in some modalities.
[00102] The 1400 process begins by identifying (in 1405) a video format that specifies a particular resolution and/or bit rate for captured video. Process 1400 then determines (at 1410) whether the format specifies temporal compression. If the format specifies temporal compression, process 1400 sets (at 1415) temporal compression parameters for the video format. In some embodiments, parameters are based on design choices made by video recording device manufacturers that are intended to maximize video quality for the particular resolution and/or bit rate. If the format does not specify temporal compression, the process proceeds to operation 1420.
[00103] In 1420, process 1400 sets the transform block size to be 8x8 for the video format. Process 1400 then sets (at 1425) multi-slice encoding to the video format. In some modalities, four slices are defined as the number of slices. However, other modalities can define a different number of slices for video images encoded using the video format. Process 1400 then sets (at 1430) the video format to examine all intra-prediction modes. After setting the video format parameters, process 1400 determines (at 1435) whether there is no more video format to set. If there are additional video formats to define, the process returns to operation 1400 to 1405 and defines another video format. Process 1400 repeats operations from 1405 to 1430 until there are no more video formats to define, after which the process ends.
[00104] The above section describes process 1400 operations performed in a particular order. However, one skilled in the art will recognize that some or all of the operations of process 1400 can be performed in any order or simultaneously. For example, operations 1420, 1426, 1430 can be performed simultaneously, operations 1425 to 1430 can be performed before operation 1420, or operations 1420 and 1430 can be performed before operation 1425. D. System
[00105] Figure 15 illustrates a block diagram of a video camera 1500 of some modalities that use the video capture, encoding and storage processes described above. Camcorder 1500 may be the same Camcorder 700, or it may differ in one or more respects. As shown in Figure 15, video camera 1500 includes an optical input 1505, an image detection circuit 1510, a video encoder 1520, and a storage device 1530. The video camera in some embodiments further includes a port. 1535 data transfer, a 1540 video decoder, a 1545 digital viewer, and a 1550 user input control.
[00106] Optical images from the outside world enter video camera 1500 through optical input 1505. In some embodiments, optical input 1505 includes an aperture and one or more optical lenses. Lenses perform focus, optical zoom or other optical processes on optical images.
[00107] An optical image coming from the optical input 1505 is projected to the image detection circuit 1510, which converts the optical image into electronic signals. In some embodiments, the image detection circuit 1510 is a charge-coupled device (CCD). A CCD includes a photoactive region that includes a two-dimensional capacitor matrix, in which capacitors accumulate electrical charges proportional to the intensity of light received. Once the matrix has been exposed to the optical image, a control circuit causes each capacitor to transfer its contents to its neighbor or to a charge amplifier, which converts the charge to a voltage. By repeating this process, the CCD samples and digitizes the optical image.
[00108] A 1520 video encoder encodes the scanned optical image. Some embodiments implement video encoder 1520 as a microprocessor to execute a set of instructions. Other embodiments implement the 1520 video encoder using one or more electronic devices, such as Application Specific Integrated Circuits (ASICs), Field Programmable Port Arrays (FPGA), or other types of circuitry.
[00109] In some embodiments, the video encoder 1520 is an MPEG-4 H.264 encoder, which uses discrete cosine transform and prediction to remove redundancies from the images. Some modalities remove both spatial and temporal redundancies, while other modalities remove only spatial redundancy or no redundancy at all. Some video encoder modalities still use entropy encoding to produce a compressed bitstream from the encoded image.
[00110] A storage device 1530 stores the encoded image. In some embodiments, storage device 1530 is a flash memory device, hard disk, or other type of non-volatile memory device capable of storing digital information such as the encoded image. The storage device is removable (eg a removable flash drive) in some modes. The stored encoded image can then be transferred out of video camera 1500 using a data transfer port 1535.
[00111] The data transfer port 1535 transfers image or other data between the storage device 1530 of the camcorder 1500 and an external device such as a computer. In some embodiments, the 1535 data transfer port uses high-throughput protocols such as Universal Serial Bus (USB) or IEEE 1394 (Fire Wire) interface to communicate with the computer. Data transfer port 1535 can also communicate with a computer using any other wired or wireless data communication protocol.
[00112] A 1550 User Input Control allows a user to adjust the settings of various 1500 video camera components. In some embodiments, the 1550 User Input Control is implemented as physical buttons on the video camera. Alternatively, or together, some modalities include a graphical interface, which allows the user to navigate through various video camera settings graphically. In some embodiments, User Input Control 1550 allows the user to adjust video decoder 1520 settings. For example, a user can set the video decoder to encode the image using any encoding modes included in the H.264 standard, or a user can set the 1520 video encoder to use only I-frames or other subsets of the H.264 standard.
[00113] Some modalities include a 1540 video decoder so that a user can view the encoded image. Video decoder 1540 is capable of decoding the image encoded by video encoder 1520 and stored in storage device 1530. In some embodiments, video decoder 1540 is part of video encoder 1520, because some embodiments of video encoder Video 1520 includes a video decoder in order to produce an encoded video stream in accordance with H.264. Digital viewer 1545 displays the video image decoded by video decoder 1540. In some embodiments, the digital viewer is implemented as part of a GUI associated with user input control 1550. II. Media editing application
[00114] Some modalities provide a media editing application for importing and editing digital video having the ability to differentiate between different incoming digital video formats. When temporally compressed digital video is imported, the media editing application transcodes the digital video and stores the encoded video in storage. When non-temporal compressed digital video is imported, the media editing application recognizes this format and stores the incoming video directly into storage without transcoding.
[00115] Figure 16 illustrates such 1600 media editing application of some modalities. Some examples of such media editing applications include iMovie® and Final Cut Pro®, both sold by Apple Inc.®. The media editing application 1600 is on a 1605 computer. In some embodiments, the 1605 computer may be a computer specifically dedicated to media editing, or it may be a computer that includes numerous other programs (eg, word processor, web browser web, computer game applications, etc.)
[00116] In addition to the 1600 media editing application, the 1605 computer also includes the 1610 manager interface and captures the 1615 module, as well as a 1620 storage device. The 1610 interface manager receives a digital video stream from a digital video source. The 1625 camera, described below, is an example of a digital video source. In some embodiments, the interface manager is an input trigger (eg FireWire input trigger, USB input trigger, etc.), which receives the video stream through a computer port (eg , a FireWire port, a USB port, etc.) that is connected to the digital video source (eg via a FireWire or USB cable, directly via a USB port, wirelessly, etc.).
[00117] The 1610 interface manager retransmits the received video stream to the 1615 capture module, which in some modalities channels the video stream from the low-level port manager (the 1610 interface manager) to the editing application of media, 1600. In some embodiments, this 1615 capture module is part of Apple Inc.® QuickTime Engine® In some embodiments, the 1615 capture module is actually a part of the 1600 media editing application. The 1620 storage stores the video received from the digital video source. The 1620 storage is part of the 1600 media editing application, in some modes too. For example, storage 1620 might be a media editing application library. In other embodiments, storage is, as shown, part of computer 1605. Storage 1620 can store more than just video clips in some embodiments. For example, storage 1620 can also store executable or other files associated with media editing application 1600 or other applications that reside on computer 1605.
[00118] Media editing application 1600 includes 1630 format recognition module, 1635 transcoder, and 1640 thumbnail generator. One skilled in the art will recognize that media editing application of some modalities will include other modules not shown in this diagram, such as editing modules, processing engine, etc.
[00119] Format 1630 recognition module receives a digital video clip from capture module 1615 after import and identifies the format of the digital video. In some embodiments, this identification determines whether the digital video is temporally compressed. The 1630 format recognition module examines digital video clip metadata in some modalities in order to identify the format (see a video structure description below for further discussion of the metadata). In some modalities, the metadata indicates whether the digital video is an iFoto format (non-temporal compressed) or a different format that uses temporal compression. In some embodiments, the format recognition module is able to identify the formats of different video clips as soon as the camera is connected to the computer 1605.
[00120] When the 1630 format recognition module identifies that the incoming digital video clip is not temporally compressed and therefore does not need to be transcoded, the 1630 format recognition module sends the video clip directly to the 1620 storage. As mentioned above, this could be the media editing application library 1600, or it could be a computer storage 1605 that is shared by multiple applications. The speed of importing such a digital video clip is tied to the video file size and the connection speed between camera and computer in some modalities, and is not tied to clip speed transcoding or clip playback. Specifically, because there is no transcoding, import speed is not tied to the processing power needed to decode and/or encode. Also, when the digital video clip is stored in the camera's random access storage, the import speed is not tied to any playback speed of the video clip which is due to reading from a tape-based storage device, which requires tape playback such that 30 minutes is required to import 30 minutes of video. Some modalities, instead of directly forwarding the video clip to storage 1620, decode the incoming video clip in order to remove spatial compression.
[00121] When the 1630 format recognition module identifies that the incoming digital video is temporally compressed, the digital video is routed to the 1635 transcoder. The 1635 transcoder, in some embodiments, decodes the digital video and re-encodes the video with only the spatial compression. Thus, the output of transcoder 1635 is non-temporal compressed video. This transcoding process generally takes substantially longer than for a non-temporally compressed video clip of equivalent length. In some embodiments, the transcoder decodes the video and does not re-encode it.
[00122] Transcoder output (non-temporal compressed video) is sent to thumbnail generator 1640 in some arrangements. The 1640 thumbnail generator generates thumbnails of each digital video image in the video clip. Thumbnails are stored in 1620 storage along with the video clip. Some modalities also send non-temporally compressed video clips arriving from the 1630 format recognition module to the 1640 thumbnail generator as an intermediate step before storage. Also, some modalities do not include a thumbnail generator and therefore do not store thumbnails with the video clip.
[00123] As mentioned above, in some embodiments the digital video stream is received from a 1625 camera. The 1625 camera may be a camera, such as the 700 digital video camera in some embodiments. The 1625 camera includes a 1645 transfer module and a 1650 video storage. The video clip storage includes numerous video clips that are stored in different formats. For example, a 1651 clip is stored in a non-temporal compressed format, clip 1652 is stored in 720p temporally compressed format, and clip 1653 is stored in 1680p temporally compressed format. As illustrated in Section T above, some modalities allow a user to select the recording format of each video clip captured by the camera. As illustrated in this figure and described below, some modalities store the video format as metadata.
[00124] The 1645 transfer module, in some embodiments, is an output conductor associated with an output port (eg a FireWire or USB) of the 1625 camera. In some embodiments, a user who interacts with the video camera via the camera user interface or the 1600 media editing application user interface (when the camera is connected to the 1605 computer) instructs the 1625 camera to transfer a specific video clip to the 1600 media editing application. clip is then transferred to computer 1605 via transfer module 1645.
[00125] Figure 16 also illustrates the 1655 video format file of some modalities that are stored in the 1625 camera. The 1655 video file is an example of a non-temporal compressed video clip. The 1655 video file includes 1660 video image data, 1665 Advanced Audio Coding (AAC) audio data, and 1670 metadata. The video image data includes the non-temporal compressed video frames in this example , and in the case of clip 1652 it would include temporally compressed video frame data. AAC 1665 audio data is a particular audio format that is required by the media editing application in some modalities. Other modalities allow for different ways of encoding audio data, such as linear pulse code modulation (LPCM) audio, for example.
[00126] As illustrated, 1670 metadata includes 1675 video format type, 1680 geotag data, a 1685 'chlorine' atom, and other 1690 metadata. The 1675 video format type indicates the encoding format of the video. That is, the 1675 format type indicates whether the video is in iPicture format (temporally compressed) and can also indicate the resolution and/or bitrate of the video. In some embodiments, media editing application 1600 requires the bit rate to be below a specific threshold to format the iframe data (eg 24 Mbps) while maintaining a particular threshold quality at a given resolution.
[00127] Geotag 1680 data, in some modalities, indicates GPS coordinates or other geographic location information about where the video clip was shot. This information is based on a geolocator module (eg a GPS receiver) in the camera. 'Chlorine' atom 1685 is used to convert correctly between different display device color spaces in some modalities. Specifically, the 'chlorine' atom indicates that the one-space conversion of a software color range should be used. The 'nclc' tag on the 'colr' atom is used in some embodiments to identify that color space conversion can go through either a software or hardware path (eg in video clip playback).
[00128] Some modalities store other 1690 metadata with the video clip as well. This metadata can include lighting information about lighting when the video was captured, cadence information, and frame rate (eg 25, 30, etc. frames per second) about the video clip, depth information of bits (eg 8 bits), etc. In some embodiments, when the video clip is transferred to the 1600 media editing application, 1670 metadata is transferred along with it and is used by the media editing application. For example, some modalities of the 1630 format recognition module determine the video format type from the 1670 metadata.
[00129] Some media editing application modalities specify requirements for non-temporally acceptable compressed video. For example, some modalities specify video encoding and compression behavior for the H.264 encoding scheme using Baseline, Main, or High profile encoding. Different profiles are different sets of capabilities in some modalities. Some modalities also specify that the entropy encoder in the camera (for example, unit 725 in Figure 7) uses either Context-based Adaptive Variable Length Coding (CAVLC) or Context-based Adaptive Binary Arithmetic Coding (CAB AC). Some modalities specify other requirements such as frame rate (eg only 25 or 30 fps), bit depth (eg 8 bits), file format (eg .mp4 or .mov), color marking (eg what kind of atom parameter 'colr' with color 'nclc' should be present, maximum bit rate (eg 24 Mbps), etc.
[00130] Figure 17 conceptually illustrates a process 1700 of some modalities for storing a video clip imported to a computer from a digital video source such as camera 1625. Process 1700 is performed by a video editing application media in some modalities (eg 1600 app). The process starts by receiving (in 1705) a video clip from the digital video source. Video clip reception can be initiated by a computer user by selecting an import option in a user interface or by dragging a video clip icon from a camera folder to a computer folder. For example, when video is stored on the random access storage camera (eg hard disk, flash memory, etc.), a user can open a folder on the computer for the video camera and see an icon for each of the video files on the camera. The user can use a cursor controller to drag the icon of a desired video clip to a folder on the computer to initiate the transfer. Video clip reception can also be automatically initiated by connecting the camera to a computer input port, etc.
[00131] The process then identifies (in 1710) the video format of the video clip. As mentioned above, in some modalities, the camcorder encodes and stores the clip in a number of different formats. For example, the video camera can encode the video clip by performing spatial compression only, or encodes the video clip by performing both spatial and temporal compression. In some embodiments, the process identifies the video format based on metadata stored in the camera and transferred with the video clip that indicates the video format. Other modalities recognize the encoding type by analyzing the video image data.
[00132] Process 1700 then determines (in 1715) whether the video is temporally compressed. This is based on the identification of the video format. When video is non-temporal compressed, the process stores (in 1720) the video clip in its native format. That is, no transcoding is needed when the video is non-temporal compressed and the video clip can be stored instantly without any processing.
[00133] When video is temporally compressed, the process transcodes (in 1725) the video to remove temporal compression. As described above, the transcoding process of some modalities decodes the video and then re-encodes the video using only spatial compression. This transcoding operation is computationally intensive and time-consuming. The process then stores (in 1730) the decoded video. After saving the video (either in native or transcoded format) 1700 then the process ends. III. computer system
[00134] Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions written to a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computational element(s) (such as processors or other computational elements such as ASIC and FPGAs), they cause the computational element(s) to perform the actions indicated in the instructions. Computer is intended in its broadest sense, and can include any electronic device with a processor. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media does not include carrier waves and electronic signals that pass wirelessly or through wired connections.
[00135] In this report, the term "Software" means the inclusion of read-only memory-resident firmware or applications stored in magnetic storage that can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as subparts of a larger program, while the software inventions remain distinct. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, software programs when installed to operate on one or more computer systems define one or more specific machine implementations that execute and perform the operations of the software programs.
[00136] Figure 18 illustrates a computer system with which some embodiments of the invention are implemented. Such a computer system includes various types of computer readable media and interfaces to various other types of computer readable media. One skilled in the art will also note that the digital video camera of some embodiments also includes various types of computer readable media. The 1800 computer system includes an 1805 bus, an 1810 processor, an 1820 graphics processing unit (GPU), an 1825 system memory, an 1830 read-only memory, an 1835 permanent storage device, an 1840 input devices, and 1845 output devices.
[00137] The 1805 bus collectively represents the entire peripheral system, and the chip set buses that communicatively connect the various internal devices of the 1800 computer system. For example, the 1805 bus communicatively links the 1810 processor with read-only memory 1830, the 1820 GPU, the 1825 system memory, and the 1835 permanent storage device.
[00138] From these various memory units, the processor 1810 retrieves instructions to execute and data to process in order to execute the processes of the invention. In some embodiments, the processor comprises a Field Programmable Gate Array (FPGA), an ASIC, or various other electronic components for executing instructions. Some instructions are passed to and executed by the 1820 GPU. The 1820 GPU can offload various computations or supplement the image processing provided by the 1810 processor. In some embodiments, such functionality can be provided using Corelmage's kernel shading language.
[00139] The 1830 read-only memory (ROM) stores static data and instructions that are needed by the 1810 processor and other computer system modules. The 1835 permanent storage device, on the other hand, is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the 1800 computer system is turned off. Some embodiments of the invention use a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 1835.
[00140] Other embodiments use a removable storage device (such as a floppy disk, flash drive or ZIP® disk, and its corresponding disk drive) as the permanent storage device. Like permanent storage device 1835, system memory 1825 is a memory read-write device. However, unlike the 1835 storage device, system memory is volatile read and write memory, just like random access memory. System memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the processes of the invention are stored in system memory 1825, permanent storage device 1835, and/or read-only memory 1830. For example, the various memory units include instructions for transforming items multimedia, according to some modalities. From these different memory units, the 1810 processor retrieves instructions to execute and data to process in order to execute processes of some modalities.
[00141] The 1805 bus also connects to the 1840 and 1845 input and output devices. The input devices allow the user to communicate information and select commands to the computer system. Input devices include 1840 alphanumeric keyboards and pointing devices (also called "cursor control devices"). The 1845 output devices display images generated by the computer system. Output devices include printers and display devices such as cathode ray tubes (CRT) or liquid crystal displays (LCD).
[00142] Finally, as shown in Figure 18, the 1805 bus also couples the 1800 computer to an 1865 network via a network adapter (not shown). In this way, the computer can be a part of a computer network (such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of networks such as like the internet. Any or all of the components of the 1800 computer system can be used in conjunction with the invention.
[00143] Some modalities include electronic components, such as microprocessors, storage, and memory that store computer program instructions on a machine-readable or computer-readable medium (alternatively known as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs ( eg DVD-ROM, double-layer DVD-ROM), a variety of rewritable/recordable DVDs (eg DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (eg SD cards , mini SD cards, Micro-SD cards, etc.), magnetic and/or solid state hard drives, read and write-only Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media and floppy disks. Computer readable media can store a computer program that is executable on at least one processor and includes instruction sets for performing various operations. Examples of hardware devices configured to store and execute instruction sets include, but are not limited to application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), programmable logic devices (PLDs), ROM, and input devices. RAM memory. Examples of computer programs or computer code include machine code as produced by a compiler and files including higher level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.
[00144] As used in this specification and any claims in this application, the terms "Computer", "server", "processor", and "memory" refer to electronic or other technological devices. These provisions exclude people or groups of people. For the purposes of the specification, the terms display or display means view on an electronic device. As used in this specification and any claims in this application, the terms "computer readable medium" and "computer readable medium" are entirely restricted to tangible, physical objects that store information in a form that can be read by a computer. These provisions exclude any wireless signals, wired download signals, and any other ephemeral signals.
[00145] Although the invention has been described with reference to numerous specific details, one skilled in the art will recognize that the invention may be incorporated in other specific forms without departing from the spirit of the invention. Furthermore, a number of the figures (including figures 8 and 11) conceptually illustrate processes. The specific operations of these processes cannot be performed in the exact order shown and described. Specific operations cannot be performed in a continuous series of operations, and different specific operations can be performed in different modalities. Furthermore, the process can be implemented using several sub-processes, or as part of a larger macro process.

权利要求:
Claims (15)
[0001]
1. Video camera (500) characterized in that it comprises: means (750) for capturing a sequence of a plurality of video images; means (515) for allowing a user to select a video encoding format from a plurality of video encoding formats including a first video encoding format for encoding the sequence of the plurality of video images using only intraframe encoding and a second video encoding format for encoding at least a portion of the sequence of the plurality of video images using interframe encoding; and means (710) for encoding the sequence of the plurality of video images in accordance with the video encoding format selected by the user.
[0002]
2. Video camera (500) according to claim 1, characterized in that the means for enabling the user to select the video encoding format comprises: means for displaying a list of the plurality of video encoding formats; and means to receive user interaction in order to select the video encoding format from the plurality of video encoding formats.
[0003]
3. Video camera (500) according to claim 1, characterized in that the first video encoding format is for encoding the sequence of the plurality of video images using only intraframe encoding at a first resolution, wherein the plurality of video encoding formats further includes a third video encoding format for encoding the sequence of the plurality of video images using only intra-frame encoding with a second resolution different from the first resolution.
[0004]
4. Video camera (500) according to claim 1, characterized in that it further comprises means to identify the selected video encoding format, retrieve a set of settings for encoding according to the video encoding format selected and sending the retrieved set of settings to the means for encoding the sequence of the plurality of video images.
[0005]
5. Video camera according to claim 1, characterized in that the means for encoding the sequence of the plurality of video images further comprises means for reducing the width of the plurality of video images before encoding the plurality of video images.
[0006]
6. Method for recording video, characterized in that it comprises the steps of: capturing a sequence of a plurality of video images; allowing a user to select a video encoding format from a plurality of video encoding formats including a first video coding format for encoding the plurality of video picture sequence using intra-frame coding only and a second video coding format for encoding at least a portion of the plurality of video picture sequence using inter-frame coding; encode the sequence of the plurality of video images according to the video encoding format selected by the user.
[0007]
7. Method according to claim 6, characterized in that the first video encoding format is an encoding scheme in which each image in the sequence of the plurality of video images is encoded without reference to any other images of the plurality of video pictures sequence and the second video coding format is an coding scheme in which at least one picture of the sequence of the plurality of video pictures is encoded with reference to at least one other picture of the sequence of the plurality of video images.
[0008]
8. The method of claim 6, further comprising (1) performing spatial compression on image data blocks and (2) when the selected video encoding format is the second video encoding format , perform temporal compression on image data blocks.
[0009]
9. The method of claim 6, further comprising storing a plurality of video encoding format settings for the plurality of video encoding formats.
[0010]
10. Method according to claim 6, characterized in that it further comprises storing the encoded sequence of the plurality of video images.
[0011]
11. Machine readable medium storing a method which, when executed by at least one processing unit, records video, the method characterized in that it comprises the steps of: capturing a sequence of a plurality of video images; a user selects a video encoding format from a plurality of video encoding formats that include a first video encoding format to encode the sequence of the plurality of video images using only intraframe encoding and a second video encoding format. video for encoding at least a portion of the sequence of the plurality of video images using intraframe encoding; encode the sequence of the plurality of video images according to the video encoding format selected by the user.
[0012]
12. Machine readable medium according to claim 11, characterized in that the method further comprises transferring the encoded sequence of the plurality of video images to an external device.
[0013]
13. Machine readable medium according to claim 12, characterized in that the external device is a computer comprising a media editing application.
[0014]
14. Machine readable medium according to claim 11, characterized in that the selected video encoding format is defined by a resolution setting that specifies at least one of 960x540 and 1280x720 and a bit rate setting that specifies up to 24 megabits per second (Mbps).
[0015]
15. Machine readable medium according to claim 11, characterized in that the step of allowing the user to select the video encoding format comprises allowing the user to select the video encoding format through a graphical interface of the user comprising a plurality of selectable user interface items (UI), each user interface item to select a specific video encoding format associated with the user interface item.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112012005549B1|2021-08-17|VIDEO CAMERA, METHOD FOR RECORDING VIDEO AND MACHINE-READABLE MEDIA FOR VIDEO EDITING AND REFORMAT FOR DIGITAL VIDEO RECORDER

---

US8731374B2|2014-05-20|Video format for digital video recorder

---

US8817873B2|2014-08-26|Image coding/recording apparatus and image coding/recording method

---

KR100987365B1|2010-10-12|Selecting key frames from video frames

---

US9832468B2|2017-11-28|Electronic device for storing image and image storage method thereof

---

US20150063439A1|2015-03-05|Encoding apparatus and method

---

US20150036739A1|2015-02-05|Method and apparatus for generating encoded content using dynamically optimized conversion

---

CN102006445A|2011-04-06|Image editing device and imaging device provided with the image editing device, image reproduction device and imaging device provided with the image reproduction device

---

JP2007074450A|2007-03-22|Encoding device and encoding method of moving image signal, and computer-readable recording medium

---

AU2014277749B2|2016-10-06|Video editing and reformatting for digital video recorder

---

KR102172995B1|2020-11-02|Electronic device for storing image and method thereof

---

JP2005176330A|2005-06-30|Image processor, control method therefor, computer program and computer readable recording medium

---

US11197014B2|2021-12-07|Encoding apparatus, decoding apparatus, and image processing system

---

JP6690536B2|2020-04-28|Image processing apparatus, image processing method, and program

---

JP2009272929A|2009-11-19|Video encoder, and video encoding method

---

BR112021013735A2|2021-09-21|IMAGE ENCODING METHOD AND DEVICE TO PERFORM MRL-BASED INTRAPREDICTION

---

JP2010148042A|2010-07-01|Recording device, recording method, and computer program

同族专利:

---

公开号 | 公开日

---

WO2011031902A4|2011-05-19|

---

EP2476256B1|2014-08-06|

---

WO2011031902A1|2011-03-17|

---

CN104952470A|2015-09-30|

---

US20110058792A1|2011-03-10|

---

JP5883474B2|2016-03-15|

---

EP2733703B1|2017-10-25|

---

AU2010292204A1|2012-05-03|

---

CN102484712A|2012-05-30|

---

JP2013504936A|2013-02-07|

---

CN104952470B|2018-02-23|

---

JP2014220818A|2014-11-20|

---

HK1169532A1|2013-01-25|

---

BR112012005549A2|2017-06-20|

---

US20140301720A1|2014-10-09|

---

JP6280144B2|2018-02-14|

---

CN102484712B|2015-05-13|

---

KR101361237B1|2014-02-11|

---

US8737825B2|2014-05-27|

---

AU2010292204B2|2014-09-18|

---

KR20120056867A|2012-06-04|

---

EP2476256A1|2012-07-18|

---

JP5555775B2|2014-07-23|

---

US9215402B2|2015-12-15|

---

EP2733703A1|2014-05-21|

---

JP2016129361A|2016-07-14|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

US5111292A|1991-02-27|1992-05-05|General Electric Company|Priority selection apparatus as for a video signal processor|

---

US5577191A|1994-02-17|1996-11-19|Minerva Systems, Inc.|System and method for digital video editing and publishing, using intraframe-only video data in intermediate steps|

---

GB2318246B|1996-10-09|2000-11-15|Sony Uk Ltd|Processing digitally encoded signals|

---

JP3787398B2|1996-11-27|2006-06-21|キヤノン株式会社|Image processing apparatus and method|

---

JP3592025B2|1997-03-11|2004-11-24|キヤノン株式会社|Captured image recording device|

---

US6115420A|1997-03-14|2000-09-05|Microsoft Corporation|Digital video signal encoder and encoding method|

---

US7110025B1|1997-05-28|2006-09-19|Eastman Kodak Company|Digital camera for capturing a sequence of full and reduced resolution digital images and storing motion and still digital image data|

---

JP3844844B2|1997-06-06|2006-11-15|富士通株式会社|Moving picture coding apparatus and moving picture coding method|

---

JP4427827B2|1998-07-15|2010-03-10|ソニー株式会社|Data processing method, data processing apparatus, and recording medium|

---

US20010004417A1|1999-12-21|2001-06-21|Ageishi Narutoshi|Video editing system|

---

US6834080B1|2000-09-05|2004-12-21|Kabushiki Kaisha Toshiba|Video encoding method and video encoding apparatus|

---

JP2003046758A|2001-07-31|2003-02-14|Canon Inc|Imaging apparatus, information processing unit, image processing unit and method therefor, and image processing system|

---

US20030123546A1|2001-12-28|2003-07-03|Emblaze Systems|Scalable multi-level video coding|

---

US7319720B2|2002-01-28|2008-01-15|Microsoft Corporation|Stereoscopic video|

---

US20070074269A1|2002-02-22|2007-03-29|Hai Hua|Video processing device, video recorder/playback module, and methods for use therewith|

---

JP3905450B2|2002-09-26|2007-04-18|富士フイルム株式会社|Digital still camera|

---

JP4015914B2|2002-09-26|2007-11-28|富士フイルム株式会社|Digital camera|

---

JP4196640B2|2002-10-21|2008-12-17|株式会社日立製作所|Data conversion method|

---

JP3750858B2|2002-11-26|2006-03-01|ソニー株式会社|Imaging apparatus and recording method|

---

JP2004187161A|2002-12-05|2004-07-02|Matsushita Electric Ind Co Ltd|Moving video data processing equipment and moving video data processing method|

---

JP3873921B2|2003-04-03|2007-01-31|ソニー株式会社|Recording apparatus and method|

---

US6958757B2|2003-07-18|2005-10-25|Microsoft Corporation|Systems and methods for efficiently displaying graphics on a display device regardless of physical orientation|

---

JP4232588B2|2003-09-19|2009-03-04|ソニー株式会社|Image processing apparatus, image processing method, and program|

---

JP2007519301A|2003-09-30|2007-07-12|ドラプレットテクノロジーインコーポレイテッド|System and method for temporal out-of-order compression and multi-source compression ratio control|

---

US7528840B1|2003-10-01|2009-05-05|Apple Inc.|Optimizing the execution of media processing routines using a list of routine identifiers|

---

JP4515886B2|2003-10-28|2010-08-04|パナソニック株式会社|In-plane predictive coding method|

---

JP3826934B2|2003-12-04|2006-09-27|コニカミノルタビジネステクノロジーズ株式会社|Image forming apparatus, image forming system, image forming method, image forming control program, and computer-readable recording medium recording image forming control program|

---

JP2005175974A|2003-12-12|2005-06-30|Canon Inc|Imaging device|

---

JP4401861B2|2004-04-30|2010-01-20|キヤノン株式会社|Image processing apparatus and control method thereof|

---

JP2005348258A|2004-06-04|2005-12-15|Canon Inc|Photographing apparatus and method, program and storage medium|

---

EP1647895B1|2004-10-14|2010-03-24|Sony Corporation|Video apparatus, video output control method, and reproduction control method|

---

US20060133476A1|2004-11-12|2006-06-22|Page Warren S|Digital in-car video surveillance system|

---

EP1806786A1|2005-02-09|2007-07-11|Matsushita Electric Industrial Co., Ltd.|Monitoring camera device, monitoring system using the same, and monitoring image transmission method|

---

JP4616057B2|2005-04-13|2011-01-19|パイオニア株式会社|Image coding apparatus and image coding method|

---

US20090123066A1|2005-07-22|2009-05-14|Mitsubishi Electric Corporation|Image encoding device, image decoding device, image encoding method, image decoding method, image encoding program, image decoding program, computer readable recording medium having image encoding program recorded therein,|

---

JP4534935B2|2005-10-04|2010-09-01|株式会社日立製作所|Transcoder, recording apparatus, and transcoding method|

---

JP2007135069A|2005-11-11|2007-05-31|Sony Corp|Imaging reproducing apparatus|

---

JP2009527135A|2006-01-05|2009-07-23|アイスポット、コーポレーション|System and method for storing, editing and sharing digital video|

---

JP2007193888A|2006-01-19|2007-08-02|Sony Corp|Recording device and method, program, coding device and method|

---

DE602007008730D1|2006-07-13|2010-10-07|Qualcomm Inc|VIDEO-CODING WITH FINE-SCALE SCALABILITY BASED ON CYCLIC-ORIENTED FRAGMENTS|

---

ES2348686T3|2006-07-13|2010-12-10|Qualcomm Incorporated|VIDEO CODING WITH FINE GRANULAR SCALABILITY THROUGH FRAGMENTS ALIGNED WITH CYCLES.|

---

JP4921076B2|2006-08-28|2012-04-18|キヤノン株式会社|Imaging apparatus and control method thereof|

---

JP2008078969A|2006-09-21|2008-04-03|Victor Co Of Japan Ltd|Moving-image coding/recording device|

---

JP2008109590A|2006-10-27|2008-05-08|Olympus Corp|Imaging system and authentication method of imaging apparatus|

---

JP2008118271A|2006-11-01|2008-05-22|Fujinon Corp|Remote control system of imaging apparatus|

---

US8116537B2|2007-04-04|2012-02-14|Sony Corporation|Image recording device, player device, imaging device, player system, method of recording image, and computer program|

---

US10454995B2|2007-06-11|2019-10-22|Crackle, Inc.|System and method for obtaining and sharing content associated with geographic information|

---

JP4786602B2|2007-06-26|2011-10-05|株式会社東芝|Information processing apparatus and program|

---

JP2009021775A|2007-07-11|2009-01-29|Panasonic Corp|Coder and coding method|

---

JP2009048469A|2007-08-21|2009-03-05|Fujitsu Ltd|Information processing apparatus and information processing method|

---

JP4922138B2|2007-11-20|2012-04-25|株式会社東芝|Moving picture coding apparatus and moving picture coding method|

---

US20090238479A1|2008-03-20|2009-09-24|Pawan Jaggi|Flexible frame based energy efficient multimedia processor architecture and method|

---

US9967590B2|2008-04-10|2018-05-08|Qualcomm Incorporated|Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter|

---

US20090320082A1|2008-06-19|2009-12-24|Iwatchnow Inc.|Method of delivering content data|

---

US20100275122A1|2009-04-27|2010-10-28|Microsoft Corporation|Click-through controller for mobile interaction|

---

US8737825B2|2009-09-10|2014-05-27|Apple Inc.|Video format for digital video recorder|

---

US8554061B2|2009-09-10|2013-10-08|Apple Inc.|Video format for digital video recorder|US8223845B1|2005-03-16|2012-07-17|Apple Inc.|Multithread processing of video frames|

---

US8554061B2|2009-09-10|2013-10-08|Apple Inc.|Video format for digital video recorder|

---

US8737825B2|2009-09-10|2014-05-27|Apple Inc.|Video format for digital video recorder|

---

US9762639B2|2010-06-30|2017-09-12|Brightcove Inc.|Dynamic manifest generation based on client identity|

---

US9838450B2|2010-06-30|2017-12-05|Brightcove, Inc.|Dynamic chunking for delivery instances|

---

AU2011201404B1|2011-03-28|2012-01-12|Brightcove Inc.|Transcodeless on-the-fly ad insertion|

---

JP5766297B2|2011-11-25|2015-08-19|三菱電機株式会社|Imaging device, video recording device, video display device, video monitoring device, video monitoring system, and video monitoring method|

---

JP2013141064A|2011-12-28|2013-07-18|Jvc Kenwood Corp|Image pickup device and control method|

---

US8488943B1|2012-01-31|2013-07-16|Google Inc.|Trimming media content without transcoding|

---

US9241167B2|2012-02-17|2016-01-19|Microsoft Technology Licensing, Llc|Metadata assisted video decoding|

---

US8855466B2|2012-05-01|2014-10-07|Eldon Technology Limited|Separate video file for I-frame and non-I-frame data to improve disk performance in trick play|

---

CN102801813B|2012-08-27|2015-11-25|广东利为网络科技有限公司|A kind of device and method automatically uploading audio/video file from terminal equipment|

---

US9112939B2|2013-02-12|2015-08-18|Brightcove, Inc.|Cloud-based video delivery|

---

WO2015050562A1|2013-10-04|2015-04-09|Intel Corporation|Technology for dynamically adjusting video playback speed|

---

US9386067B2|2013-12-30|2016-07-05|Sonic Ip, Inc.|Systems and methods for playing adaptive bitrate streaming content by multicast|

---

US9911460B2|2014-03-24|2018-03-06|Microsoft Technology Licensing, Llc|Fast and smart video trimming at frame accuracy on generic platform|

---

CN106856571B|2015-12-09|2019-12-06|阿里巴巴集团控股有限公司|Transcoding method and device|

---

CN105681787B|2016-01-22|2019-01-18|北京大学|The method and apparatus that space-time signal is encoded|

---

CN109155838A|2016-05-13|2019-01-04|深圳市大疆创新科技有限公司|System and method for video to be presented by transcoding|

---

KR20180036153A|2016-09-30|2018-04-09|주식회사 요쿠스|Video editing system and method|

法律状态:
2018-03-27| B15K| Others concerning applications: alteration of classification|Ipc: H04N 7/00 (2011.01), G11B 27/00 (2006.01) |

---

2019-01-08| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|

---

2020-03-03| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2021-04-06| B06A| Patent application procedure suspended [chapter 6.1 patent gazette]|

---

2021-07-06| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|

---

2021-08-17| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 09/09/2010, OBSERVADAS AS CONDICOES LEGAIS. PATENTE CONCEDIDA CONFORME ADI 5.529/DF, QUE DETERMINA A ALTERACAO DO PRAZO DE CONCESSAO. |

优先权:

---

申请号 | 申请日 | 专利标题

---

US24139409P| true| 2009-09-10|2009-09-10|

---

US61/241,394|2009-09-10|

---

US12/636,699|2009-12-11|

---

US12/636,699|US8554061B2|2009-09-10|2009-12-11|Video format for digital video recorder|

---

US12/781,813|2010-05-17|

---

US12/781,813|US8737825B2|2009-09-10|2010-05-17|Video format for digital video recorder|

---

PCT/US2010/048324|WO2011031902A1|2009-09-10|2010-09-09|Video editing and reformatting for digital video recorder|

---